Introduce support for MSC4429: Profile Updates for Legacy Sync by anoadragon453 · Pull Request #19556 · element-hq/synapse

anoadragon453 · 2026-03-13T20:57:08Z

This PR implements support for MSC4429: Profile Updates for Legacy Sync.

~~Recommended to review commit-by-commit.~~ Commit history has become messy, suggest reviewing as a single item.

Paired with matrix-org/complement#849 and branch https://github.com/matrix-org/sytest/tree/anoa/msc4429

Tracking issue for removing unstable identifiers in Synapse: #19891

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct (run the linters)

Copilot

Pull request overview

Implements experimental MSC4429 support by introducing a new profile_updates stream (persisted + replicated) and surfacing profile field changes in legacy /sync when enabled.

Changes:

Add profile_updates stream tracking (DB schema, stream token field, notifier + replication stream plumbing).
Emit MSC4429 profile updates in /sync responses and add filter support for selecting profile fields.
Record profile field updates on profile mutations (set/unset displayname/avatar/custom fields, deactivation cleanup).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/rest/client/test_rooms.py	Updates hardcoded stream tokens to match the new `StreamToken` shape.
tests/rest/admin/test_room.py	Updates hardcoded stream tokens to match the new `StreamToken` shape.
synapse/types/init.py	Adds `PROFILE_UPDATES` stream key and extends `StreamToken` serialization/parsing.
synapse/streams/events.py	Includes `profile_updates_key` in current/bounded stream tokens.
synapse/storage/schema/main/delta/94/01_profile_updates_seq.sql.postgres	Adds Postgres sequence for the new stream.
synapse/storage/schema/main/delta/94/01_profile_updates.sql	Adds `profile_updates` table + indexes.
synapse/storage/schema/init.py	Bumps schema version to 94 and documents the change.
synapse/storage/databases/main/profile.py	Adds profile update persistence/query APIs and stream ID generator wiring.
synapse/rest/client/versions.py	Advertises `org.matrix.msc4429` in `unstable_features`.
synapse/rest/client/sync.py	Encodes MSC4429 profile update payload into `/sync` response.
synapse/replication/tcp/streams/_base.py	Adds `ProfileUpdatesStream` replication stream.
synapse/replication/tcp/streams/init.py	Registers `ProfileUpdatesStream` for replication.
synapse/replication/tcp/handler.py	Replicates `ProfileUpdatesStream` from configured writers.
synapse/replication/tcp/client.py	Handles incoming profile update replication and notifies listeners.
synapse/notifier.py	Allows `StreamKeyType.PROFILE_UPDATES` to wake `/sync` waiters.
synapse/handlers/sync.py	Generates initial + incremental MSC4429 profile update sections in `/sync`.
synapse/handlers/profile.py	Records profile update markers on profile mutations and notifies.
synapse/config/experimental.py	Adds `msc4429_enabled` config flag.
synapse/api/filtering.py	Adds `/sync` filter support for selecting profile fields (MSC4429-gated).
docker/complement/conf/workers-shared-extra.yaml.j2	Enables MSC4429 in complement worker test config.
changelog.d/19556.feature	Adds changelog entry for MSC4429 experimental support.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Sytest changes for element-hq/synapse#19556

We create a stream table dedicated to tracking profile updates. The sync machinary can this stream to understand whether a profile update needs to be included in a client's sync response.

Add a new stream type for profile updates. This allows sync processes to determine which profile updates a given user has or hasn't seen yet.

Replicate changes to the profile updates stream so that sync workers know there's a new profile update, and wake up sync waiters accordingly. Updates are keyed by `room_id` - sync waiters should only wake up if a user they share a room with updates their profile.

Based on the provided since token and filter.

And update relevant documentation/integration test config.

@user

Prevent the `ProfileRestServlet` from handling requests intended for the `ProfileFieldRestServlet`. If a requester tried to call PUT `.../profile/@user:domain/(key_id?)` a worker that did not mount `ProfileFieldRestServlet`, they would then receive a `405 Method Not Allowed` instead of the expected `404 Not Found`.

… leave Regardless of action. This is cleaner and also ensures a join/leave/join/leave case might not get confused.

…oined profile updates rows, due to membership event changes

… values change

To make CI happier, even though we just want to compare content has changed.

Extracted to a follow-up pr. This reverts commit b657d2f

Setting a profile field (including displayname / avatar_uri) now needs to happen on a profile updates stream writer. There are new "dispatch" methods to do the profile field set/delete, that handle pushing the update over replication, if required. Profile field updates and profile updates stream writes are now in the same database transaction.

…ds, to update the profile updates stream

…ile fields, to update the profile updates stream" This reverts commit 1937e76.

I changed things so deactivation did call these and no tests failed.

We now do all the actions relating to the stream updates in one transaction. Additionally ensure we wake up the notifier for the users concerned.

…rofile updates stream updates in a single transaction

…e right instance, if necessary

# Conflicts: # synapse/rest/client/versions.py

Since c63d77a moved things

…e not in any rooms

Thanks sytest 🤩

jaywink · 2026-07-08T10:25:27Z

I think this is ready for another round of review 🙏 A big thanks @reivilibre for all the detail in the previous reviews.

There is one unresolved comment: #19556 (comment) - I'm unsure what is the right way to go here, but have suggested my favourite option I could come up with.

Replication path arguments need to be strings.

Since we moved a lot of stuff to happen in the profile update stream writers, we need to always have the replication endpoints enabled now.

Add experimental feature flag msc4429_enabled

632c83e

anoadragon453 requested a review from Copilot March 17, 2026 11:56

Copilot started reviewing on behalf of anoadragon453 March 17, 2026 11:57 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread synapse/replication/tcp/client.py Outdated

Comment thread synapse/storage/schema/main/delta/94/01_profile_updates.sql Outdated

Comment thread synapse/storage/databases/main/profile.py Outdated

Comment thread synapse/handlers/sync.py Outdated

anoadragon453 force-pushed the anoa/msc4429 branch 2 times, most recently from 83e699f to a846945 Compare March 17, 2026 16:21

anoadragon453 added a commit to matrix-org/sytest that referenced this pull request Mar 17, 2026

Allow stream_writer to write to the profile_updates stream

0b0ab30

Sytest changes for element-hq/synapse#19556

anoadragon453 added a commit to matrix-org/sytest that referenced this pull request Mar 17, 2026

Allow stream_writer to write to the profile_updates stream

ffdc272

Sytest changes for element-hq/synapse#19556

anoadragon453 mentioned this pull request Mar 17, 2026

Allow stream_writer to write to the profile_updates stream matrix-org/sytest#1420

Open

anoadragon453 force-pushed the anoa/msc4429 branch from dc8e9f8 to a309372 Compare March 18, 2026 15:58

anoadragon453 marked this pull request as ready for review March 18, 2026 15:58

anoadragon453 requested a review from a team as a code owner March 18, 2026 15:58

MadLittleMods added the A-Profiles label Mar 18, 2026

anoadragon453 added 10 commits March 24, 2026 13:10

Schema for new profile updates stream table

d0b616d

We create a stream table dedicated to tracking profile updates. The sync machinary can this stream to understand whether a profile update needs to be included in a client's sync response.

Add profile updates stream

63cfbe7

Add a new stream type for profile updates. This allows sync processes to determine which profile updates a given user has or hasn't seen yet.

Implement replication

e45045d

Replicate changes to the profile updates stream so that sync workers know there's a new profile update, and wake up sync waiters accordingly. Updates are keyed by `room_id` - sync waiters should only wake up if a user they share a room with updates their profile.

Allow fetching latest changes via /sync and filtering

d9adfd8

Based on the provided since token and filter.

Add msc4429_enabled to Complement tests

30e8737

newsfile

cfb7034

Update unit tests with new token format

5b4d505

Constrain profile endpoint handling to profile_updates writer workers

86572c6

And update relevant documentation/integration test config.

Correct the Complement worker endpoint configuration

f27ca70

anoadragon453 force-pushed the anoa/msc4429 branch from a309372 to f27ca70 Compare March 24, 2026 13:11

anoadragon453 mentioned this pull request Mar 25, 2026

Remove redacted_because from internal unsigned. #19581

Merged

Half-Shot mentioned this pull request Mar 31, 2026

MSC4429: Profile Updates for Legacy Sync matrix-org/matrix-spec-proposals#4429

Open

MadLittleMods added the A-Sync label Apr 6, 2026

reivilibre requested review from reivilibre and removed request for a team April 7, 2026 16:20

erikjohnston reviewed Apr 9, 2026

View reviewed changes

Comment thread docs/upgrade.md Outdated

Comment thread synapse/handlers/sync.py Outdated

Comment thread synapse/handlers/sync.py Outdated

jaywink added 4 commits July 6, 2026 15:08

Clear all old ProfileUpdate rows from profile_updates_per_user on a…

b571fce

… leave Regardless of action. This is cleaner and also ensures a join/leave/join/leave case might not get confused.

Add a test to ensure display name changes don't generate extra room_j…

f431951

…oined profile updates rows, due to membership event changes

Clarify clear_profile_updates_for_user docstring

ac1bdfb

Add docstring to get_lazy_loaded_profile_fields_cache

73986ef

dbkr reviewed Jul 6, 2026

View reviewed changes

Comment thread synapse/handlers/sync.py

Ensure we bust the lazy loading cache for profile updates, when field…

10cccac

… values change

github-advanced-security AI found potential problems Jul 6, 2026

View reviewed changes

Comment thread synapse/handlers/sync.py Fixed

jaywink and others added 17 commits July 6, 2026 18:42

Use sha256 instead of md5

15466d2

To make CI happier, even though we just want to compare content has changed.

Revert "Add a prune job for the profile updates stream tables"

2bea8c0

Extracted to a follow-up pr. This reverts commit b657d2f

Update check_schema_delta.py to better parse raw SQL schema deltas

aa0dbdc

Make delete_profile_upon_deactivation first delete the profile fiel…

1937e76

…ds, to update the profile updates stream

Remove a few unused methods and fix docstrings

9052265

Revert "Make delete_profile_upon_deactivation first delete the prof…

d0aa8d0

…ile fields, to update the profile updates stream" This reverts commit 1937e76.

Add some notes on missing tests for future readers

d7e0d35

I changed things so deactivation did call these and no tests failed.

Refactor user left room / user joined room profile updates

7398242

We now do all the actions relating to the stream updates in one transaction. Additionally ensure we wake up the notifier for the users concerned.

Ensure delete_profile_upon_deactivation does profile deletion and p…

ae4e686

…rofile updates stream updates in a single transaction

Dispatch deletion of profile upon deactivation over replication to th…

ef9d995

…e right instance, if necessary

Merge remote-tracking branch 'origin/develop' into anoa/msc4429

cb81bd3

# Conflicts: # synapse/rest/client/versions.py

Update Rust side versions endpoint

3180924

Since c63d77a moved things

Adjust a few new tests after merging from develop

2ed4201

Ensure our own syncing user always gets their updates, even if they'r…

cb322e4

…e not in any rooms

Fix deactivation from a worker that is not profile writer

e9b8fd7

Thanks sytest 🤩

Add include_profile_updates_in_sync to the config schema

3ed759a

jaywink requested review from dbkr and reivilibre July 8, 2026 10:23

jaywink added 3 commits July 8, 2026 15:15

Hopefully final fix to deactivate over replication

50527b1

Replication path arguments need to be strings.

Rename profile updates db deltas as we got new stuff from develop

fcac746

Always enable profile replication endpoints

8e75e7b

Since we moved a lot of stuff to happen in the profile update stream writers, we need to always have the replication endpoints enabled now.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce support for MSC4429: Profile Updates for Legacy Sync#19556

Introduce support for MSC4429: Profile Updates for Legacy Sync#19556
anoadragon453 wants to merge 141 commits into
developfrom
anoa/msc4429

anoadragon453 commented Mar 13, 2026 •

edited by jaywink

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jaywink commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Uh oh!

Conversation

anoadragon453 commented Mar 13, 2026 • edited by jaywink Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jaywink commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

anoadragon453 commented Mar 13, 2026 •

edited by jaywink

Loading